Rhythmic Gesticulator

Automatic synthesis of realistic co-speech gestures is an increasingly important yet challenging task in artificial embodied agent creation. Previous systems mainly focus on generating end-to-end manner, which leads to difficulties mining the clear rhythm and semantics due complex subtle harmony between speech gestures. We present a novel gesture method that achieves convincing results both semantics. For rhythm, our system contains robust rhythm-based segmentation pipeline ensure temporal coherence vocalization explicitly. semantics, we devise mechanism effectively disentangle low- high-level neural embeddings motion based linguistic theory. The embedding corresponds while low-level relates variations. Lastly, build correspondence hierarchical motion, resulting rhythm- semantics-aware synthesis. Evaluations with existing objective metrics, newly proposed rhythmic metric, human feedback show outperforms state-of-the-art by margin.